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SUMMARY 


The  purpose  of  thire-  paper  is  to  illustrate  how 


the  functional  equation  technique  of  the  theory 


of  dynamic  programming  may  be  employed  to  treat 
a  class  of  variational  problems  with  constraints./ 
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NOTES  IN  THE  THEORY  OP  DYNAMIC  PROORAMMINO  -  IV: 
A  VARIATIONAL  PROBLEM  WITH  CONSTRAINTS 

by 

Richard  Bellman 


§1 .  Introduction 

We  have,  in  another  place,  [5]*  considered  the  problem  of 
maximizing  the  functional 

(1.1)  j(y)  -  y'J  P(x.y)dt 
subject  to  the  constraints 

(1.2)  a.  dx/dt  =  0(x,y),  x(0)  =  c, 
b.  0  <  y  £  x, 

using  the  classical  techniques  of  the  calculus  of  variations, 
with  modifications  Imposed  by  the  constraint  (1.2b). 

We  now  wish  to  treat  the  problem  using  the  approach  of  the 
theory  of  dynamic  programming.  Expositions  of  the  techniques  we 
shall  employ  may  be  found  in  [l] ,  [2]  or  [3],  and  we  refer  to 
these  works  for  more  detailed  discussion. 

The  basic  idea  is  to  consider  a  variational  problem  as  a 
multi-stage  decision  process  of  continuous  type.  The  emphasis 
will  now  be  upon  determining  y(0)  as  a  function  of  c  and  T  above, 
rather  than  detc  •mining  y  as  a  function  of  t  for  0  ^  t  ^  T. 
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We  shall  show  how  we  may  obtain  a  functional  equation  for 
U(c,T)  -  Max  J(y), 

y 

and  how  a  limiting  forr.  of  this  functional  equation  may  be  used 
to  determine  the  structure  of  the  extremal  curve  given  certain 
simple  structural  properties  of  F(x,y)  and  G(x,y). 

In  order  to  establish  these  results  rigorously,  we  consider 
first  the  finite  analogue  of  the  variational  problem  above, 
namely,  the  problem  of  maximizing 

N 

(1.3)  F<v  yk)< 

1(^0 

where  the  y^  are  subject  to  the  constraints 

(1.4)  a.  x,^  -  Xfc  -  Q(x^,  yk),  k  -  0, 1 ,2, . . .  ,N-1 , 

*>•  0 1  yk  ^  v 

This  we  again  attack  by  means  of  the  functional  equation 
approach,  employing  a  technique  we  have  used  repeatedly,  cf.  [l] , 
[2],  l3j •  This  approach  is  particularly  suitable  for  a  machine 
computation  of  the  problem. 
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$2.  The  3aslc  Functional  Equation 

Let  us  make  the  assumption  that  the  solution  to  the  varia¬ 
tional  problem  described  in  (l.l)  and  (1.2)  exists  and  that  y(t) 
is  a  continuous  function  of  t  in  [o,T^  for  every  T  ^  0. 

We  then  define  the  function 

(2.1)  U(c,T)  -  Max  J(y) . 

y 

Let  us  now  proceed  formally  to  obtain  a  partial  differential 
equation  for  U  on  the  assumption  that  U  has  continuous  partlals 
with  respect  to  c  and  T. 

We  have,  along  an  extremal, 

(2.2)  U(c,3-T)  -  F(  x,  y)<dt  *  y^S+T  F(x.y)dt. 

Hence,  employing  the  principle  of  optimality",  the  section  of 
the  function  y(t)  in  [0,s]  is  determined  by  the  equation 

(2.3)  U(e,s-T)  -  Max  C  yS  F(x,y)dt  +  /:S',T  F(x,y)dt  ]  . 

YL0,5j 

Consider  the  second  integral  over  [S,S-*Tj  .  At  t  *  S,  the 
value  of  x  is  x(S)  a3  determined  by  the  differential  equation  of 
(l.2a).  Since  the  problem  is  independent  of  the  start '.nr  time, 
we  see  that 


(2.4)  y^S+T  F( x,y)dt  =  U(o(S),T). 
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Hence,  (2.3)  becomes 

(2.5)  U(c,svr)  -  Max  ,  [  jTZ  r(x,y)dt  +  U(c(S),T)  ]. 

y.o.s]  1/0 

If  we  now  l*-t  S — >  +  0  and  assume  the  continuity  of  all  functions 
appearing,  we  obtain  in  the  limit  the  Dartial  differential  equation 

(2.6)  UT  =  Max  [  P(c,v)  +  0(c,v)U  j, 

'  0<v^c  c 

where  we  set  v  -  y(0)  for  typographical  convenience.  The  con¬ 
dition  0  ^  v  <  c  is  the  consequence  of  the  restriction  0  ^  y  ^  x. 

$3*  Heuristic  Considerations 

Let  us  now  see  what  we  can  deduce  from  (2.6)  under  the 
assumption  that  U  is  a  positive,  increasing  function  of  T. 

This  is  a  very  natural  condition  to  expect  to  be  satisfied  in 
a  number  of  problems  of  engineering  and  economic  origin. 

We  first  of  all  impose  the  conditions  that  F(x,y)  and 
0(x,y)  be  concave  functions  of  y  for  all  x  2  y  2  follows 

that  the  function 

(3-1)  K(v)  =  F(c,v)  *  0(c,v)Uc 

for  any  fixed  values  of  c  and  U  will  be  a  concave  function  and 
possess  a  single  maximum,  which  may  be  at  \  -  C,  v  -  c,  or  In 
between. 

If  we  assume  that  U  is  a  continuous  function  of  T,  as  we 

C 

have  every  right  to  expect,  we  obtain  the  important  result  that 
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a.8  T  varies  a  region  where  v  *  0  cam  never  abut  a  region  where 
v  =  c.  There  must  always  be  a  transition  region  where  0  <  v  <  c, 
a  region  we  call  the  Euler  region  since  the  Euler  equation  must 
be  valid  there,  a  region  of  free  variation. 

The  critical  value  of  v  is  the  point  where  K'(v)  *  0,  or 

(3.2)  Fv  +  Gv  Uc  =  0. 

Let  us  make  the  further  assumption  that  0y  >  0  for  all  v  in  the 
range  0  £  v  £  c  for  any  c  >  0.  If  we  assume  that  U  — >  ®  as 
T — *  CD ,  again  a  physically  plausible  result  in  many  of  these 
problems,  it  follows  that,  for  sufficiently  large  T,  K'(v)  = 

Pv  +  °v  Uc  wil1  be  Positive  for  0  ^  v  £  c.  The  maximum  will  then 
occur  at  v  =  c. 

It  is  easy  to  show  that  Uc-»0  as  T — ►().  If  we  assume  that 
Pv  <  0  for  a11  0  v  i  c»  as  is  the  case  in  the  case  F(x,y)  =  x  -  y 
treated  below,  we  see  that  K'(v)  <  0  for  small  T  and  hence  the 
maximum  is  at  v  =  0. 

Under  the  assumption  that  Uc  is  positive  and  monotone 
increasing  in  T,  we  see  that  the  solution  will  be  the  following: 

(3.3)  v  -  0,  0  ^  c  i  T! 

0  <  v  <  c,  Tj  <  T  <  T2 

T2  £  T, 


v  -  c. 
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where  the  transition  points  and  T2,  In  general,  will  be 
functions  of  c,  and  may  degenerate,  i.e.  =  0,  -  T2  or 

T2  -  T. 

We  see  then  the  utility  of  the  functional  equation  aa  a 
heuristic  device  for  obtaining  information  concerning  the  struc¬ 
ture  of  the  solution. 

To  prove  these  results,  we  have  two  avenues  open  to  us.  We 
may  first  of  all  use  the  method  of  successive  approximations  on 
(2.6),  obtaining  a  sequence  of  functions  Un(c,T)  in  the  following 
way.  A  choice  of  vq(c,T)  subject  to  the  inequality  0  ^  vQ(c,T)  ^  c, 
yields  a  function  Uq(c,T)  determined  by  the  partial  differential 
equation 

UoT  ’  P<c'vo)  +  0(c'vo)Uoc’ 

UQ(c,0)  ■  0  for  c  ^  0. 

Having  obtained  UQ,  we  choose  a  new  v^(c,T)  as  the  function 
yielding  the  maximum  of 

K(v,UQ)  -  F(c,v)  +  0(c,v)UQC 

subject  to  0  £  v  £  c.  The  choice  of  v1  yields  in  turn  a 
function  U1  as  the  solution  of 

U1T  -  Kfvj.Uj) 

U.(c,0)  -  0  for  c  2  0. 
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We  now  continue  in  this  fashion. 

We  shall  In  a  subsequent  paper,  in  a  aeries  of  which  M 
Is  the  first,  consider  in  detail  the  existence  and  uniqueness 
of  solutions  of  functional  equations  of  the  form  of  (2.6)  and 
the  convergence  of  the  successive  approximations  above. 

Here  we  shall  follow  a  different  path  and  consider  the 
discrete  multi-stage  process  corresponding  to  the  variational 
problem.  As  we  noted  above,  a  derivation  of  these  results 
employing  classical  techniques  is  contained  in  [5]- 

$4.  Finite  Version 

Replacing  the  integral  by  a  Riemann  sum  and  the  differ¬ 
ential  equation  by  a  difference  equation,  we  are  led  in  a  natural 
way  to  consider  the  problem  of  determining  the  maximum  of 

N 

(4.1)  J({yk})  -  £ 

where  x,  and  y,  are  related  by  the  equations 

(4.2)  ^  —  Xy  *  3  ( ,y^) ,  k  *  0,1,2,..., N— 1 , 

xo  =  c' 

and  y,  is  restricted  by  the  condition, 

0  S  'J\:  ii  *  0,1,2,  •  .  •  ,N. 


(4.3) 
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assume  that  P  and  Q  are  continuous  functions  of  x  and  y, 

and  set 

(*•*)  UN(c)  -  Max  J((y  } )  . 

{y*} 

The  basic  functional  equation  Is  now 

UN+l(c>  "0<^J  |><c.v)  +  Un(c+G(c,v))]  ,  N  -  0,1,2,... 

using  the  notation  v-y  ,  w  tn 

o 

(4.6)  U  (c)  -  Max  F(c,v). 

0<v<c 

In  order  to  simplify  the  analysis  we  shall  consider  only 
the  case  where 

(4 .7)  ?(c,v)  •  c-v  , 

a  case  of  some  Interest  lr  lt3elf. 

It  corresponds  to  a  multi-stage  allocation  process  where 
we  have  a  single  resource,  measured  by  x^  at  time  k.  A  certain 
quantity  of  this  resource,  y^,  may  be  used  to  Increase  xR  yielding 

(4‘8)  xk+l  =  xk  +  °^xk*yk) 

at  time  k+1:  The  profit  on  the  other  hand  will  be  measured  by 
xk  “  yk‘  The  Pr°blem  Is  to  determine  the  allocation  policy  which 
maximizes  the  total  M-atage  profit,  cf.  [l]  ,  [2]  ,  for  a  discussion 
of  similar  problems . 
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55-  A  Simple  Case 

Let  us  begin  with  a  simple  case,  where  the  analysis  will  not 
overshadow  the  ideas.  The  discrete  version  is  the  problem  of 
maximizing 

(5-D  J({yk}>  -  U*-yk>. 

where 

(5-2)  (a)  *k+1  -  xk  +  b(yk) 

(b)  0  <  yk  <  xk,  k  -  0,1,2, ...N  . 

Setting 

».<«)  -  Max 

{yk>  1  J 

we  clearly  have  the  functional  equation 

(5-5)  UN  ,(c)  -  Max  [c-v+UM(c+b(v))l  ,  N  -  1,2,... 

0<  v<c  L  "  J 

U0(°)  *  c- 

Our  aim  is  to  determine  the  structure  of  the  optimal  policy 
under  appropriate  assumptions  concerning  b(y).  We  shall  assume 

(5.4)  (a)  b(0)  =  0,  b '  (0)  -  od 

(b)  b'(y)  >  0,  b'(y)  0  as  y  ao 

(c)  b" (y )  <  0  . 

A  3lmple  function  satisfying  these  conditions  is  b(y)  -  yV *  . 
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We  shall  show  that  for  each  N,  the  optimal  first  alloca¬ 
tion  Vjj(c)  has  the  following  fora  as  a  function  of  c, 

(5.5)  v„(c)  .  o.  0 

o  <  v||(c)  <  °»  °M  <  0 

Wher*  {°n}  18  8  *«Qu«nce  we  shall  determine  Inductively 
below.  In  this  case*  there  le  only  one  transition  point. 

The  proof  will  be  Inductive.  Let  ue  begin  with  the  case 

N-l. 

We  have 

(5.6)  U,(c)  -  Max  [  c-v  ♦  U  (c+b(v))  ]  -  Max  [G  (v,cfl 

0<v^c  °  J  0^v<c  °  J 

If  an  Internal  maximum  occurs.  It  occurs  at  a  point  where 


This  equation  Is 

<5-7)  ~T7\  "  Uo  (c+b(v))  . 

b  (v) 

Since  Uc(c)  -  c,  UQ(c)  •  1,  equation  (5  - 7 )  by  virtue  of  the 
assumptions  concerning  b  has  precisely  one  positive  root.  If 
this  root  Is  less  than  c,  there  Is  an  Internal  maximum.  The 
critical  value  of  c  13  then  given  by  the  root  of 

(3.8)  - 1; 

b'  (c) 


call  this  value  c 


P—640 

-11- 


If  c  <  c i ,  the  maximum  in  (5.6)  will  occur  at  v-c .  If 
c  >  Cj,  there  will  be  an  Internal  maximum  at  vt. 

Hence ,  we  have 

(5-9)  U 1  ( c )  -  UQ(c+b(c)),  for  c  <  ci,  v-c, 

-  c— vj  +  UQ(c+b(vi)),  for  c>c,  v-vi<c. 

The  function  Ui(c)  la  clearly  differentiable  for  c  <  Ci 
and  for  c  >  cx.  We  have 

(5.10)  Ul(c)  -  (l+b'(c))  Uq  ( c  +b ( c ) )  -  1  +  b 1 (c ) ,  c  <  ct, 

-  1  +  Uq  (c+b(c ) )  -2,  c  >  ct  . 

At  c-ci,  b'(c)  -  1,  and  we  have  equality.  Hence,  Ui(c)  has  a 
continuous  derivative.  Furthermore,  Ul  (c)  >U^(c)  for  c  >  0. 
Let  us  now  Investigate  convexity.  We  have 

(5.11)  U'i(c)  -  b"(c)  ,  c  <  Cl 

-  0  ,  c  >  ci 

Hence  u'i(c)  <  0  for  all  c  >  0. 

We  now  turn  to  the  case  N  -  2 . 

N  -  2  We  have 

(t>.12)  Ua(c)  -  Max  [c— v  +  U 1  (c+b( v ) )]  . 

O^v^c 

If  an  Internal  maximum  occurs.  It  occurs  at  the  point  where 

(5-13)  — -  -  Ul  (c+b(v)). 

b  ’  ( v ) 
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*  It 

Since  Ut  (c+b(v))  £  0  and  (  (v j )  >  0,  there  Is  precisely  one 

.  .  ii 

root  of  this  equation,  which  we  call  va  -  va(c).  Since  Ui  >  UQ, 

It  follows  that  va(c)  >  Vi(c). 

As  above,  the  critical  value  of  c  Is  the  root  of 

(5.14)  Fr^y  -  u;  (c*b(c)). 

Call  this  root  ca.  Since  Ut  >  UQ  and  ( 1/b * )  >0,  ca  >  c». 

We  thus  have 

(5.15)  Ua(c)  -  Uj (c+b(c ) ) ,  for  c  <  ca 

■  c-va  ♦  Ut  (c+b(va ) ) ,  for  c  >  ca 

Furthermore,  we  have 

(5.16)  Ua(c)  -  C  l+b'(c) 3u|(c+b(c)),  for  c  <  ca 

-  l+Uj(c+b(va))+^  C  — 1+b'  ( v 2 ) U |  (c+b(va  ) )  J 

-  1+UJ  (c+b(va ) )  -  l-*-l/bi  (Va)»  for  c  >  ca  • 

At  c»ca,  U'(c)  Is  continuous,  recalling  (5.14). 

2 

Let  us  now  examine  the  convexity  of  Ua(c). 

(5.17)  Ua(c)  -  b"(c)ul  (c+b(c))  ♦  C  1+b'  (°)  J2  Ut(c+b(c)),  for  c<ca 

-  u;(c+b(va),:  l-*-b'(va)^2-3  ,  for  c  >  ca . 

Using  (5.13), 

(5.18)  _  -_<>’(»,)  .  u'(c+b(v»))Cl*b'(».)  U*-:  . 

C  O' (»,)]* 
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or 

(5.19)  dva  [b'(y2)uj(c+b(v2))  +  —  ]  -  -U'i' (c+b(v8 )  )  . 

Hence,  <  0*  Returning  to  (5.18)  thin  yields  1+b' (v8  )^2- >  0. 
Using  this  In  (5-17),  Ug(c)  <  0  for  c  >  c8.  Since  (5-17)  shows 
that  U8(c)  <  0  for  0  <  c  <  c2 ,  we  see  that  U8(s)  <  0  for  all 
c  >  0,  but  Is  not  continuous  at  c  -  c8. 

Now  to  the  final  3tep  that 

(5.20)  U^(c)  >  U|(c)  . 

Having  established  this,  we  have  all  the  Ingredients  of  an  induc¬ 
tive  proof. 

To  establish  this  Inequality,  we  must  consider  three  distinct 
intervals  ,c8]  ,  and  [c8»<»3. 

In  [C^cj],  we  have 

(5.21)  U’(c)  -  [  1  +  b'(c)  ^  U'(c+b(c))  >  [  1  +  b'(c)  3  U'(c+b(c)) 

2  1 

-  U'(c)  . 

i 

In  [[  c8 ,  qd  J  ,  we  have 

(5.22)  U£(o)  -  1  +  1A'(v2)  >  1  +  lA'iVj)  -  U£(c)  . 

Since  v*  >  Vj.  The  remaining  interval  to  consider  is  [ci,c*].  In 
[ci,cj  we  have 


(5.23)  UJ(c)  -  1  -*•  lA'(vi) 

U^(c)  -  [  1  4  b'(c)  ]  U* (c  +  b(c ) )  . 
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Slnce  va  »  c  In  Ect»Ca3» 

(5.24)  -  1  +  b'(v)U|(c  +  b(v))  £  0 
for  0  <  v  <  c8.  In  particular, 

(5.25)  b'(c)lP(c  +  b(c) )  >  1. 

We  now  wish  to  show  that 

(5.26)  U|(c  +  b(e))  £  l/b'Ort) 

for  Ci  <  c  £  ca.  Since  l/b'(v)  Is  Increasing  and  Vj(c)  £  c, 
lA'(v»)  <  l/b'  (c).  By  (5-25)  It  follows  that,  for  c  <  ca, 

(5.27)  Uj(c  +  b(c))  >  l/b’(c)  £  lA'(vi)  . 

We  now  have  all  the  material  required  for  an  Inductive  proof 
of  the  following 

(5.28)  Theorem.  For  each  N  there  exlatai  a  function  vN (c )  with 
the  following  properties : 

(a)  vN(c)  1®.  monotone  decreasing  as  c  Increases; 

(k)  VN+1^^  ^  ^  ■  1,2,... 

(c)  There  Is  a  unique  solution  of  v^c)  «*  c  which  we 

can  cN.  and  cN+1  >  cN; 

(d)  For  0  <  c  <  cN,  we  have  UN(c)  -  un_i(c  +  b(c)); 

(e)  For  cN  <  c,  nave  UN(c)  =*  c_vN(c ) fUN_1 £  cfb(vN(c))  3; 


(f)  UN'(c)  >  Un_l’(c)>  N  * 


c  >  0; 
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(g)  Uj(c)  £  0,  N  -  1,2,...,  c  £  0  . 

J6.  A  More  General  Problem 

Let  us  now  consider  the  problem  of  maximizing 

(!)  J(y)  ■  ^/JT(::-y)dt 

subject  to  the  relation 

(2)  al-o(x.y).  x(o)  -  c  , 
and  the  constraint 

(3)  0  <  y  <  x  . 

As  above,  we  begin  by  considering  the  discrete  version  of  the 
problem,  where 

(4>  JC{yk}J  -^Io  <*k-yk)  * 

and 

(5)  (a)  xk+1  -  +  0(v»k)‘  k  -  °’1’2 . 

(b)  o  <yx  <\  ■ 

Setting 

(6)  UN(c)  -  Max  J[  {yk}3 

y 


we  clearly  have 
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(7)  U,(c)  -  e  , 

W  =  )  3  ?lax  n  c-v  *D.(c+C(c,v))]  ,  N  -  1,2,. .. 

°<V£C 

Our  aim  is  to  determine  the  structure  of  the  optimal  policy 
under  appropriate  assumptions  concerning  G(c,v).  This  is  equiva¬ 
lent  to  determining  the  structure  of  v  =  vN(c)  as  a  function  of  c 
and  N. 

We  shall  assume  that  G(c,v)  satisfies  the  following  conditions 

(8)  (a)  Gv(c,v)  >  0,  Gy(c,v)  ->OaBv^oo,  Gv(c,v)  -»  od  as 

v  0,  uniformly  in  c. 

<b)  0C  >  0,  G„  S  0. 

( c )  [  l/Ov(c,v)  3V“C  is  monotone  Increasing  in  c. 

(d)  r*Gcc  +  2rsGcv  +  saGyv  is  a  negative  definite  form. 

(e)  [  1  +  Gc(c,v)  3  /Gy(c,v)  is  monotone  increasing  In  v.* 

As  above,  we  shall  employ  an  Inductive  approach. 

31,  The  Case  N-2 
We  have 

(1)  U a (c )  -  Max  C  c-v  +  U  x  (c+G(c ,v) )  J 

0£v£c 

Let  us  set 

(2)  P»(**e)  •  c-w  i(o+0(c,v))  . 

If  internal  maximum  occurs  In  (l),  it  occurs  at  a  point  where 

*  In  the  limit  of  the  continuous  case,  this  requires  that  1/0  (c,v) 
be  monotone  increasing  In  v,  which  Is  a  consequence  of  (d).v 
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(4)  I/O  (c,v)  =  U'(c+G(e,v)  -  1 

*  4 

By  virtue  of  our  assumptions  concerning  G,  this  equation  has  pre— 
cisely  one  root  for  any  value  of  c.  If  this  root  is  less  than  c, 
there  is  an  internal  maximum. 

The  critical  value  of  c  lc  then  given  by  the  root  of 

(5)  [  l/0v(c.v)  ]v=c  »  1  . 

Let  us  call  this  value  ci.  Cur  assumption  ensures  that  this  equation 
has  precisely  one  root. 

If  c  <  c»,  there  will  be  no  internal  maximum  and  the  maximum 
will  occur  at  v  ■  c.  If  c  >  Ci,  there  will  be  an  internal  maximum 
at  a  value  v  <  c  which  we  call  v*  »  Vi(c). 

Hence  we  have 

(6)  U2(c)  -  U|(c+0(c,c))  .  c  <  ct  f 

C— V|40  4  (c+G(c  #Vi  )  )  ,  C  >  Cj  , 

The  function  f*(c)  is  clearly  differentiable  for  c  <  c»  and 
c  >  c i .  We  have 

(7)  IP  (c)  -  1  +  (Gc(c,c)  +  [Qv(c,v)  ]v.c)  #  c  <  Ci 

«  1  +  (1  +  Ge(c,v»))  ,  c  >  Ci  . 

For  further  use,  we  note  that  (4)  shows  that  we  may  write 


(3)  ^ 
«iv 


(8)  IP(c)  -  1  ♦  (1  +  Gc(c,Vi))/Gv(c,vt)  , 


c  >  Ci 
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At  c  »  clt  v»(c)  •  c  and  [  Gy(c,v)  3V-C  “  Hence  we  have  equality 

of  right  and  left  hand  derivatives  of  U2(c)  at  c  •  c,.  Thus  U »(c) 

e 

Ip  contlnucur  for  all  veluee  of  c. 

Since  Uj(c)  =  1,  (7)  shows  that 

(9)  U'(c)  >  U  '  (c) 

£  1 

for  all  c  >  0. 

We  now  wish  to  show  that  Us (c)  la  monotone  Increasing,  which 
la  to  Bay  thatUs(c)  Is  concave.  To  acconpllah  we  shall  prove  a 
useful  lemma. 

-^8.  A  Lemma  on  Concavity 
let  us  prove 

Lemma  1.  Let  $(x,y)  be  a  concave  function  of  x,y  for  x,y  >  0f 

(1)  +(  Axi+(1-  ^)x?,  ^yt+d-  *)y»)  >  Aiit(xi,y,)  ♦  (1-  >)+(xa,ya) 

for  0  <  A  <1. 

Then 

(2)  f(x)  -  Max  i(x,y) 

o<y<x 

Is  concave. 

Proof:  If  y  -  x,  we  have  f(x)  ■  ^(x,x),  clearly  concave.  If  y  ■  0, 
f(x)  -  i(x,o ),  also  clearly  concave.  If  0  <  y  <  x,  then  y  la  deter¬ 
mined  by 


(?)  ty(x,y)  -  0 
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Then 

(*>  f(x)  -  4>x  +  6y  -  *x  • 

Thus 

(5)  ?"(x)  »  +  *xy 

^xr.^yy  ~  **xy 

*yy 

using  (*)  which  yields  $Xy  +  $yy  dy/dx  -  0 

The  condition  that  f(x,y)  be  concave  is  equivalent  to 

<6)  ^Kx  +  2rE  Ky  +  s“*yy 

being  negative  definite.  Hence  the  right  side  of  the  equation  in 
(3)  is  negative. 

Let  us  now  show  that 

(7)  ?(o,v)  -  c-v  *  U( j(c ,v ) ) 

is  concave  if  U  is  concave  and  monotone  increasing  and  G  is  concave. 
We  have 

(8)  Fcc  “  GccU'(G)  -  °cU"(G) 

Fvv  "  GvvU’(G)  + 

Fcv  'W™  +  W'<G> 


From  this  it  is  clear  that  F(c,v)  is  concave. 
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Comblnlng  these  results  we  see  that  a(c)  Is  concave. 

59*  The  Case  N-5 
We  have 

(1)  Us(c)  -  Max  [  c-v+Oa(c-fO(c#v))  J 

0£v<c 

-  Max  C  Pa(v,c)  3 
0£v<c 

If  an  Internal  maximum  occurs.  It  occurs  at  a  point  where 

(2)  l/Ov(c,v)  -  ui(c+Q(c,v)) 

Since  l/Gy  Is  monotone  Increasing  In  v,  and  Ua(c+G(c,v) )  Is  monotone 
decreasing  In  v,  there  Is  exactly  one  root  of  this  equation  which  we 
call  va  -  va(c).  Since  Va  >  0 1  It  Is  clear  that  va(c)  >  vt(c). 

The  critical  value  of  c  Is  the  root  of 

(5)  [  1/Gv(c»v)  -  oJ(c4.G(c,c))  . 


Call  this  root  ca>  It  Is  clear  that  c8  >  C|. 


Prom  (2)  we  see  that  dv/dc  Is  negative. 
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Ref  erring  to  the  figure  above  It  Is  clear  that  c2  >  Cj.  Hence 

(4)  U3(c)  -  U2(c+0(c.c))  ,  c  <  c2 

-  c-v2+U2(c+0(c,v2))  ,  c  >  c2  . 

Thus 


(5)  u^(c)  =  {l-[  Qc(c,c)  +  C  Qv(c,v)  Jv=c  U'2(c+0(c,c)).  c  <  c, 

°  1  +  (l+0c(c,v2)U^(c+0(cfv2)),  c  >  c2 
,  .  1+0c<c'v2> 

*  1  +  — w  r.  ..  i 


The  concavity  of  U3(c)  follows  as  before. 

That  U^(c)  >  U£(c)  Is  clear  for  c  >  c2  and  0  £  c  £  c^  It 
remains  to  prove  that  the  Inequality  holds  In  Cc1#c23*  After 
having  established  this,  we  have  all  the  material  for  an  Induc¬ 
tive  proof  of  the  structure  of  vN(c). 

In  C  oltc2J  we  have 


(6)  U£(c)  =  1  +  (l+Oc(c,v1))U^(c+G(c,v1) 

U$(c)  -  [l+(0c(c,c)+[0v(c,v)]v<iC)]u^(c+0(c,c)) 

The  fact  that  v2  -  c  In  Q  c1#c2  3  implies  that 

(7)  -1  +  OvU£(c+Q)  ^  0 

for  0  £  v  £  c  and  Cj  £  c  £  c2.  Hence 

(8)  ■■■(-)  ^[lt  °C(C-C)  *  C  °y(c.  v)  Jv.c  ] 

3v  ^  r  n  T 
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^  1  + 


i  +  ac(c,c) 
Cov(c,v)  Jv-C 


On  the  other  hand 


(9)  U£(c) 


1  + 


l+Oc(c,v1) 

°v(c»v1) 


Since  v.(c)  ^  c  for  c.  ^  c  <  cd#  and  Q  1  +  0  fc,v)  1/0  (c,v)  qua 
function  of  v  Is  monotone  Increasing,  by  assumption,  we  see  that 
U^(c)  >  U£(c).  We  now  have  all  the  material  required  for  an  Induc¬ 
tive  proof. 


Summarizing  our  results  we  see  that  we  have  a  monotone  In¬ 
creasing  sequence,  £0^  ,  possessing  the  property  that 

(10)  Vj|  -  c  ,  0  £  c  i  cN 

*  vlj(c)  <  c  »  Cjj  <  c  <  ao  . 

Each  function  UM(c)  Is  monotone  Increasing  In  c  and  concave  and  the 
sequence  ^U£(c)^  Is  monotone  Increasing  In  N. 

This  property  carries  over  In  the  limit  as  the  discrete  process 
goes  over  to  the  continuous  and  enables  us  to  use  the  functional 
equation  to  determine  the  properties  of  the  solution.  It  may  be 
p^-v^d  by  a  straightforward  argument  that  the  limit  of  the  discrete 
c* ae  la  Indeed  the  continuous.  We  shall  omit  the  proof  here. 


i 
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